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DETAILED ACTION 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 28-55 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

For claim 28, the claimed module as disclosed on page 17 of the specification is 
a computer algorithm that is not embodied on a computer readable medium nor 
executed by a computer processor. 

For claim 42, the machine readable medium is not a computer readable medium, 
the machine readable instructions are not computer readable instructions, and the 
machine is not a computer. This claim should recite a "computer readable medium 
storing computer readable instructions which, when executed by a computer processor, 
cause the computer processor to perform a method comprising...." 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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Claims 1-3, 11, 16-17, 28-29, 32, 37, 42-43, 46, 51 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Schultz et al., "Subpixel Motion Estimation for Super- 
Resolution Image Sequence Enhancement" published by the Journal of Visual 
Communication and Image Representation, v 9 n 1, pages 38-50, March 1998 
("Schultz") in view of U.S: Patent Number 7,088,773 issued to Paniconi et al. 
("Paniconi"). 

For claims 1, 28 and 42, Schultz discloses computing a respective motion map 
for each pairing of a reference image and a respective image neighboring the reference 
image in a sequence of base images, each motion map comprising a set of motion 
vectors mapping reference image pixels to respective neighboring image pixels (page 
38, motion vectors are estimated between video frames, where the accuracy of the 
estimated motion fields has a direct influence on the quality of the high resolution video 
still image); 

Schultz discloses the target image having a target resolution level and the base 
images having a base resolution level equal to or lower than the target resolution level 
(abstract, a high resolution video still image is estimated from several low resolution 
frames); and 

Schultz discloses computing pixel values for the target image based on 
corresponding pixel value contributions from the base images (see for example the 
Bayesian multiframe resolution enhancement discussed in section 2). 

Paniconi discloses assigning respective regions of a target image to motion 
classes based on the computed motion maps and computing pixel values for the target 
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image based on corresponding pixel value contributions from the base images selected 
in accordance with the motion classes assigned to the target image regions (column 2 
lines 39-49, methods for motion segmentation can divide a frame into a number of 
motion classes, where each moving object is assigned to its own motion class). 

It would have been obvious to one of ordinary skill in the art to include the motion 
classes of Paniconi with the super resolution enhancement algorithm of Schultz, 
because each class can be tracked across frames using vectors, which saves 
processing time as taught by Paniconi. 

For claim 2, Schultz discloses generating for each image pair respective dense 
motion vectors describing motion at pixel locations with respective sets of parameters in 
a motion parameter space at sub-pixel accuracy (the entire article starting with the title 
and ending with the conclusion discusses this claimed element). 

For claims 3, 29 and 43, Paniconi discloses assigning pixels of the reference 
image to respective motion classes and up-projecting the motion class assignments to 
pixels of the target image (see for example figures 1a, 1b and 1c which show classes 
projected to pixels, see also column 6 lines 1-24). 

For claims 1 1 , 32 and 46, Schultz discloses computing an alignment accuracy 
map for each pairing of the reference image and a respective neighboring image based 
on the computed motion maps (page 43, the displaced frame difference along with its 
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mean and variance are computed to determine how well the motion vectors have been 
estimated). 

For claim 16, Schultz discloses up-projecting the motion maps from the base 
image resolution level to the target image resolution level (see page 45, the up-sampled 
frames produce the up-sampled subpixel resolution motion vectors). 

For claims 17, 37 and 51, Schultz discloses re-mapping the neighboring images 
to the reference frame of the up-projected reference image using the respective up- 
projected motion maps (page 45, re-mapping is performed by computing the Bayesian 
multiframe HRVS for each motion field). 

Claims 4-5, 30 and 44 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schultz and Paniconi as applied to claims 1-3 above, and further in 
view of U.S. Patent Number 6,269,175 issued to Hanna et al. ("Hanna"). 

For claims 4, 30 and 44, Paniconi discloses computing motion magnitude maps 
from each motion map (see columns 8-9 and figures 6a, 6b, 6c, 6d, 6e and 6f, although 
Paniconi does not use the words "motion magnitude maps," he clearly computes the 
motion magnitude map for each class from the motion map as shown for example in the 
discussion of figure 6c), Hanna discloses multiresolution image pyramid representations 
that can down-sample the computed motion magnitude maps of Paniconi to a pyramid 
of motion magnitude maps at respective resolution levels lower than the base resolution 



Application/Control Number: 10/824,692 Page 6 

Art Unit: 2624 

level, and segment pixels in the pyramid of down-sampled motion magnitude maps into 
motion classes. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to include the multiresolution image pyramid resolutions with the image 
enhancement and segmenting methods of Schultz and Paniconi, because the pyramid 
resolutions provide the benefit of very efficient computation even when large 
displacements are present but also provide subpixel accuracy in displacement 
estimates as taught by Hanna at column 6 lines 40-50. 

For claim 5, Paniconi discloses segmenting, and Hanna discloses iteratively 
estimating pixels in the pyramid of motion magnitude maps from a coarsest resolution 
level up to the base resolution level, wherein estimating from each coarser resolution 
level is up-sampled to initialize the estimation at a finer resolution level, and refined by 
further estimation at the finer resolution level. 

Claims 6, 12-15, 31, 33-36, 45, 47-50 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Schultz and Paniconi as applied to claims 1-3 above, and 
further in view of Eren et al., "Robust, Object-Based High-Resolution Image 
Reconstruction from Low-Resolution Video," published by IEEE Transactions on Image 
Processing, v 6 n 10, October 1997 ("Eren"). 

For claims 6, 31 and 45, Eren discloses generating a separate motion class 
segmentation map for each pairing of the reference image and a respective neighboring 
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image and merging the separate motion class segmentation maps into a unified motion 
class segmentation map for the reference image (see pages 1448-49 and figures 1(a) 
and (e), the segmentation maps of each frame are given or can be computed, and the 
segmentation map of a reference frame is given. The maps are used to create a high 
resolution mosaic). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the enhancement methods of Schultz and Paniconi with the 
segmentation map of Eren to enable object based processing that uses more accurate 
motion models and improves the quality of the reconstructed image as taught by Eren in 
the abstract. 

For claims 12, 33 and 47, Eren discloses re-mapping neighboring images to a 
coordinate frame of the reference image using respective motion maps, and computing 
correlation measures between pixels of the reference image and pixels of each of the 
motion-compensated neighboring images (see section III robust high-resolution 
reconstruction using a validity map, the correlation measure is determined over a 3x3 
window of pixels between the reference image and other low resolution images). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the enhancement methods of Schultz and Paniconi with the validity 
map of Eren to improve the quality of the image by using projections only for those pixel 
locations for which the motion vectors are accurate as taught by Eren. 
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For claims 13, 34 and 48, Eren discloses up-projecting the computed alignment 
accuracy maps from the base image resolution level to the target image resolution level 
as discussed in section III. 

For claims 14, 35 and 49, Eren discloses up-projecting the motion maps from the 
base image resolution level to the target image resolution level, and classifying motion 
vectors in each up-projected motion map into valid and invalid motion vector classes 
based on the up-projected alignment accuracy maps as disclosed throughout the entire 
document's discussion of the validity map. 

For claims 15, 36 and 50, both Eren and Schultz disclose values of target image 
pixels with corresponding pixels in all neighboring images being associated with motion 
vectors in the invalid motion vector class are computed by interpolating up-projected 
pixel values of the reference image. 

Claims 7-10, 18-27, 38-41, 52-55 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schultz, Paniconi and Eren as applied to claim 6 above, and further 
in view of U.S. Patent Numer 6,307,560 issued to Kondo et al. ("Kondo"). 

For claim 7, Paniconi discloses that reference image pixels are respectively 
assigned to a motion class. Kondo discloses that the motion class is selected from a 
motion class set including a high motion class and a low motion class, and motion 
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vectors assigned to the high motion class have higher magnitudes than motion vectors 
assigned to the low motion class (column 5 lines 18-36). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to include the low and high motion classes of Kondo with the motion classes 
of Paniconi for the benefit of varying the description of the motion according to the 
application as taught by Kondo in column 5 line 25. 

For claim 8, Paniconi discloses assigning a given reference image pixel to the 
low motion class in the unified motion class segmentation map when the given pixel is 
assigned to the low motion class in all of the separate motion class segmentation maps, 
and assigning a given reference image pixel to the high motion class in the unified 
motion class segmentation map when the given pixel is assigned to the high motion 
class in any of the separate motion class segmentation maps (see figures 1a, 1b and 1c 
each object has its own class that is tracked over multiple frames). 

For claim 9, Kondo discloses that the motion class set further includes an 
intermediate motion class, and motion vectors assigned to the intermediate motion 
class have magnitudes higher than motion vectors assigned to the low motion class and 
lower than motion vectors assigned to the high motion class (see column 5). 
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For claim 10, Kondo discloses assigning a given reference image pixel to the 
intermediate motion class (column 5) and Paniconi and Eren disclose the unified motion 
class segmentation map. 

For claim 18, Paniconi discloses regions of the target image are respectively 
assigned to a motion class selected from a motion class set (abstract) and Kondo 
discloses the motion classes include a high motion class and a low motion class, and 
motion vectors of regions assigned to the high motion class have higher magnitudes 
than motion vectors of regions assigned to the low motion class (column 5) 

For claims 19, 38 and 52, Eren suggests computing a pixel-wise combination of 
pixel value contributions from the up-projected reference image and the re-mapped 
neighboring images weighted based on pixel-wise measures of alignment accuracy 
between the reference image and the corresponding neighboring images (the validity 
map and blurring is accounted for to determine whether pixels are aligned and 
contribute). 

For claims 20, 39 and 53, Eren suggests that pixel value contributions from the 
re-mapped neighboring images are additionally weighted based on measures of 
temporal distance between the reference image and the corresponding neighboring 
images (the temporal distance is accounted for using motion vectors and the 
segmentation maps). 
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For claims 21 , 40 and 54 Eren suggests classifying low motion class reference 
image pixels and their corresponding pixels in the neighboring images based on 
measures of local texture richness (see page 1449 the background is low motion that is 
classified based on texture). 

For claim 22, Eren and Paniconi suggest low motion class reference image pixels 
and their corresponding pixels in the neighboring images are quantitatively evaluated for 
local texture richness, and are classified into a texture class selected from the texture 
class set including a high texture region class and a low texture region class, and pixels 
assigned to the high texture region class have higher local texture measures than pixels 
assigned to the low texture region class (see page 1448 of Eren and the abstract of 
Paniconi, the low motion classes can be segmented into low motion background and 
low motion object based on the textures). 

For claim 23, Eren suggests values of target image pixels classified into the low 
texture region class in the reference image and all of the respective neighboring 
images, are computed by interpolating up-projected pixel values of the reference image 
(the pixel values of the background are determined from the reference image). 

For claim 24, Eren and Paniconi suggest a value of a given target image pixel 
classified into the high texture region class in the reference image or any respective 
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neighboring images is computed based on a pixel value contribution from the up- 
projected reference image, and a pixel value contribution from a given re-mapped 
neighboring image weighted based on a measure of local texture richness computed for 
the given pixel, a measure of motion estimation accuracy computed for the given pixel, 
and a measure of temporal distance of the neighboring image from the reference image 
(the pixel values of a moving object are determined from texture, motion, and temporal 
images). 

For claims 25, 41 and 55, Eren discloses values of target image pixels are 
computed based on pixel value contributions from a number of base images 
neighboring the reference image, and Paniconi discloses the number of neighboring 
base images being different for different motion classes (if an object is moving faster 
than another object, it will usually be present in fewer base images before it moves out 
of the field of view). 

For claim 26, the combination of Eren, Paniconi and Kondo suggests values of 
target image pixels assigned to the high motion class are computed based on pixel 
value contributions from a fewer number of neighboring base images than the values of 
target image pixels assigned to the low motion class (slower objects are present in more 
images than faster objects). 
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For claim 27, Kondo suggests an intermediate motion class, the motion vectors 
of regions assigned to the intermediate motion class have lower magnitudes than 
motion vectors of regions assigned to the high motion class and higher magnitudes than 
motion vectors of regions assigned to the low motion class, and values of target image 
pixels assigned to the intermediate motion class are computed based on pixel value 
contributions from a fewer number of neighboring base images than the values of target 
image pixels assigned to the low motion class but a higher number of neighboring base 
images than the values of target image pixels assigned to the high motion class. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffrey S. Smith whose telephone number is 571 270- 
1235. The examiner can normally be reached on M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571 272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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June 4, 2007 




